Put this aside from TestClientSideEncryptionProse so Setenv() can be used without being influenced by t.Parallel().

The IsExpired logic returns true when credentials are nil, which means the provider will always be considered expired before the first retrieval. This forces an immediate retrieval, but if ExpirationCallback is also nil after the first retrieval, it will always return true, causing credentials to be re-fetched on every call. Consider returning a.credentials == nil when ExpirationCallback is nil to avoid unnecessary re-fetching after initial retrieval.

The closure over fn in the loop may capture the wrong value if the loop variable is reused. While this loop only processes the 'aws' key specifically, consider using a function parameter or declaring fn within the block to ensure correct closure capture: provider := fn before the closure.

The closure over fn in the loop may capture the wrong value if the loop variable is reused. While this loop only processes the 'aws' key specifically, consider using a function parameter or declaring fn within the block to ensure correct closure capture: provider := fn before the closure.

Thanks for this work @qingyang-hu! Before committing to a detailed review, I'd like to align with the team on the overall approach for providing custom credentials by clarifying that there are (at least) two approaches:

1. The callback approach suggested in this PR
2. The submodule to use the official aws-sdk-go-v2 as prescribed in GODRIVER-3567 and POC’d in PR #2057

Here is how I anticipate folks would use the submodule approach (see here for required POC updates):

// Custom credentials provider using the aws-sdk-go-v2
myProvider := aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
    return aws.Credentials{
        AccessKeyID:     "USER_ACCESS_KEY",
        SecretAccessKey: "USER_SECRET_KEY",
        SessionToken:    "USER_SESSION_TOKEN",
        Source:          "CustomProvider",
    }, nil
})

// Load AWS config with custom credentials provider using the aws-sdk-go-v2
awsCfg, _ := config.LoadDefaultConfig(
    ctx,
    config.WithCredentialsProvider(myProvider))

opts := options.Client().ApplyURI(uri)

// Extend options to include awsConfig using mongoaws
opts = mongoawsv2.WithAWSConfig(opts, awsConfig)

// Use with MongoDB
client, _ := mongo.Connect(opts)

The pros of the submodule solution:

• Ideal for power users
• Zero AWS dependencies in the Go Driver while still getting official AWS SDK support
• The solution is future proof in that we can add mongoawsv3, mongoawsv4, etc. whenever needed
• Much better user experience than having to define an abstract credential object that could apply to any provider
• We can deprecate and no longer add to the AWS legacy code.

The Cons of the submodule solution:

• We have to maintain the legacy behavior until v3
• Requires an extra import when working with AWS

A potential concern is that the opaque any return type reduces compile-time type safety, but this is validated at connection time via mongo.Connect(), which is acceptable.

If we think custom credentials are rarely needed (power users only), then I suggest prioritizing GODRIVER-3567 (submodule approach) to avoid committing AWS-specific types to the stable API. Power users who need custom credentials likely already use AWS SDK v2 in their applications, so the dependency is acceptable. This prevents long-term technical debt from maintaining options.Credentials.

Otherwise, if credentials have broader appeal or users want to avoid AWS SDK v2 dependency, then this PR is acceptable imo. However, if we merge, I suggest we prioritize implementing GODRIVER-3567 afterward and clearly document it as the preferred approach for users already using AWS SDK v2 by deprecating options.Credentials.

IIUC, the SetCredentialProviders portion for CSFLE provides value regardless of which path we choose for connection authentication.

CC: @matthewdale

Addressing the pros:

Ideal for power users

The user experience for both the callback and submodule approach are very similar.

Zero AWS dependencies in the Go Driver while still getting official AWS SDK support

That is true for the callback approach as well. Dependency management for the callback approach is arguably simpler because you don't have to worry about what version of the AWS SDK the "mongoawsv2" submodule depends on.

The solution is future proof in that we can add mongoawsv3, mongoawsv4, etc. whenever needed

I think we can address future AWS SDK changes in either approach. The structure of AWS credentials and credentials providers has barely changed over the lifetime of both AWS SDK major versions.

Much better user experience than having to define an abstract credential object that could apply to any provider

I agree with this. We should make the credential and credential provider types in this PR specific to AWS.

We can deprecate and no longer add to the AWS legacy code.

There are two sections of AWS-specific code in the Go Driver:

1. Code to support the server's "MONGODB-AWS" auth API.
2. Code to fetch AWS credentials from different environments.

Section 1 is specific to the MongoDB server API and is something we will have to continue maintaining as long as the server supports it, whether in the main Go Driver module or in an AWS-specific submodule. Section 2 is what we want to delegate to the AWS SDK.

Addressing the cons:

We have to maintain the legacy behavior until v3

I think this is true for either approach.

Requires an extra import when working with AWS

I agree with this con.

E.g. user experience using the callback approach (with the Go Driver types renamed to be AWS-specific):

myProvider := aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
	return aws.Credentials{
		AccessKeyID:     "USER_ACCESS_KEY",
		SecretAccessKey: "USER_SECRET_KEY",
		SessionToken:    "USER_SESSION_TOKEN",
		Source:          "CustomProvider",
	}, nil
})

// Load AWS config with custom credentials provider using the aws-sdk-go-v2
awsCfg, _ := config.LoadDefaultConfig(
	context.Background(),
	config.WithCredentialsProvider(myProvider))

opts := options.Client().
	ApplyURI("mongodb://localhost:27017").
	SetAuth(options.Credential{
		AWSCredentialsProvider: func(ctx context.Context) (options.AWSCredentials, error) {
			cfg, err := awsCfg.Credentials.Retrieve(ctx)
			if err != nil {
				return options.AWSCredentials{}, err
			}

			return options.AWSCredentials(cfg), nil
		},
	})
client, _ := mongo.Connect(opts)

Overall, I think the callback approach requires fewer moving pieces and has an almost identical user experience. Also, it seems to align a little better with the API in other drivers (e.g. mongodb/node-mongodb-native#4373) considering DRIVERS-3194 was closed as a duplicate of DRIVERS-2903. I think we should move forward with the callback approach, amending this PR to make the credentials and credentials provider types AWS-specific.

@matthewdale

The user experience for both the callback and submodule approach are very similar.

I see it a bit differently. For example, if a caller needs to inject AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE to obtain credentials from the EKS Pod Identity Agent, the callback approach would require you the user to

1. Read both the relative URI and token file from environment variables
2. Make the HTTP request with the Authorization header set to the token value
3. Parses the ECS response and returns credentials in the format the driver expects
4. Pass this function to AwsCredentialsProvider (for auth) or SetCredentialProviders (for encryption)

// callback for to obtain credentials from the EKS Pod Identity Agent
func awsCredentialProvider(ctx context.Context) (options.Credentials, error) {
    awsCfg, err := config.LoadDefaultConfig(ctx)
    if err != nil {
        return options.Credentials{}, err
    }
    
    creds, err := awsCfg.Credentials.Retrieve(ctx)
    if err != nil {
        return options.Credentials{}, err
    }
    
    return options.Credentials{
        AccessKeyID:     creds.AccessKeyID,
        SecretAccessKey: creds.SecretAccessKey,
        SessionToken:    creds.SessionToken,
        Source:          creds.Source,
        CanExpire:       creds.CanExpire,
        Expires:         creds.Expires,
    }, nil
}

Shipping a standalone implementation upfront gives power users a supported hook, which is a much better UX:

// Load AWS config with default credential chain
// This automatically supports:
// - Environment variables
// - AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
// - AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE (automatically read and sent)
// - EC2 instance metadata
// - ECS container credentials
// - etc.
awsCfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
    panic(err)
}

opts := options.Client().ApplyURI(uri)

// Extend options to include awsConfig using mongoawsv2
opts = mongoawsv2.WithAWSConfig(opts, awsCfg)

// Use with MongoDB - the AWS SDK will handle the token file automatically
client, err := mongo.Connect(opts)
if err != nil {
    panic(err)
}

That is true for the callback approach as well. Dependency management for the callback approach is arguably simpler because you don't have to worry about what version of the AWS SDK the "mongoawsv2" submodule depends on.

That's a fair point about avoiding a direct dependency. However, I'm concerned that copying SDK code (e.g., a signer) introduces security risks: if the SDK maintainers patch a CVE, we might not catch it in time and could continue shipping a vulnerable implementation. By relying directly on the SDK, we can benefit from their security updates automatically. Do you see a way to mitigate that risk with the current implementation?

We should make the credential and credential provider types in this PR specific to AWS.

From what I've seen, it seems like we'd be building and maintaining AWS-specific logic that essentially acts as a shim around the SDK. Is there a precedent or pattern you're pulling from that makes this the preferred approach? It seems like just relying on the SDK seems far more prudent.

Section 1 is specific to the MongoDB server API and is something we will have to continue maintaining as long as the server supports it

That's the longterm (v3) goal of GODRIVER-3567 and PR 2057, to containerize that logic into its own submodule, which would allow us to remove the AWS v1 legacy code from the driver. I realize this needs better documentation. I think this approach could give us a cleaner separation and make future maintenance easier. Does that address your concern?

Also, it seems to align a little better with the API in other drivers (e.g. mongodb/node-mongodb-native#4373) considering DRIVERS-3194 was closed as a duplicate of DRIVERS-2903

To clarify: DRIVERS-3194 was closed as a duplicate specifically because a submodule solution was expected to satisfy the requirements of DRIVERS-2903. The submodule approach was the intended path forward for addressing both tickets.

The callback solution is fair, but I think we should give strong consideration to the SDK solution as our end goal. If we implement the callback and then agree that the SDK solution is long-term better, we'd essentially need to turn around and deprecate it once we ship the submodule which feels like unnecessary churn.

To recap, my main concerns with the callback approach:

• We miss upstream CVE patches if we copy SDK code (not huge, given users can update Go toolchain locally)
• Power users still need workarounds for cases like EKS Pod Identity to avoid complex overhead
• We're already removing v1 AWS code (GODRIVER-3570), so this adds another deprecation cycle

For other drivers:

• Node uses an opt-in dependency for aws4 for signing, but plans to copy signing into the driver to reduce their overall dependency count as part of their serverless driver initiative, since the SigV4 signing algorithm is straightforward and spec-defined.
• Python relies on the SDK for all aspects of AWS Auth, but it is an opt-in dependency
• .NET also uses a standalone module that takes a dependency on the official AWS SDK, and will be fully integrated with this solution as of SHARP-4390

We should rename this to AWSCredentials since it seems to be specific to the MONGODB-AWS auth mechanism.

We should rename this to AWSCredentialsProvider since it seems to be specific to the MONGODB-AWS auth mechanism.

Initialisms in Go are conventionally all caps. This should be AWSCredentialsProvider.